Skip to content

feat(llm): add OrcaRouter as a named OpenAI-compatible gateway - #205

Merged
LauraGPT merged 3 commits into
modelscope:mainfrom
XiaoHuo888-hue:add-orcarouter-provider
Aug 27, 2026
Merged

feat(llm): add OrcaRouter as a named OpenAI-compatible gateway#205
LauraGPT merged 3 commits into
modelscope:mainfrom
XiaoHuo888-hue:add-orcarouter-provider

Conversation

@XiaoHuo888-hue

@XiaoHuo888-hue XiaoHuo888-hue commented Aug 21, 2026 •
edited by LauraGPT
Loading

Copy link
Copy Markdown

Summary

Adds OrcaRouter as a named, OpenAI-compatible smart-routing gateway for FunClip's transcript-driven LLM clipping — mirroring the existing atlascloud/ and minimax/ gateway routes.

Users can now select any orcarouter/ model in the LLM Model Name dropdown (orcarouter/auto, orcarouter/fusion, orcarouter/fusion-flash, orcarouter/fusion-mini), paste an OrcaRouter API key in the APIKEY box, and run 'LLM Inference'. FunClip sends the transcript and prompts to https://api.orcarouter.ai/v1/chat/completions, and the returned segments work with the existing 'AI Clip' button unchanged.

  • funclip/llm/openai_api.py: orcarouter/ prefix branch in _resolve_model_config() → ORCAROUTER_API_BASE (https://api.orcarouter.ai/v1, overridable via ORCAROUTER_API_BASE) + ORCAROUTER_API_KEY env fallback. Model IDs are sent verbatim with the orcarouter/ prefix, as the gateway routes by namespace.
  • funclip/launch.py: SUPPORT_LLM_PREFIX + dispatch branch + 4 gateway model dropdown entries.
  • tests/test_orcarouter_api.py, tests/test_orcarouter_launch_integration.py: unit tests mirroring the MiniMax/AtlasCloud conventions.
  • README.md / README_zh.md: "Using OrcaRouter as your LLM gateway (optional)" section.

It also offers gateway-level, zero-trust controls for AI agents on the same endpoint. Enforcement follows the Guardrail or Firewall policy attached to the relevant scope for prompts, responses, and tool calls that traverse the gateway; selecting the documented tight posture enables default-deny behavior. Enforcement is opt-in: the integration supplies only the base URL, API key, and model, and does not attach a policy by itself. Apply a policy via the security quickstart; see the Guardrails and Firewall docs for how coverage is scoped.

User impact

FunClip users get a one-dropdown way to route LLM-assisted clipping through a single gateway that can switch routing targets (frontier and open-weight models) without touching FunClip configuration.

Validation

  • I ran the relevant tests or commands:
    • python3 -m pytest -q tests/test_openai_api.py tests/test_minimax_api.py tests/test_minimax_launch_integration.py tests/test_orcarouter_api.py tests/test_orcarouter_launch_integration.py → 18 passed
    • python3 -m py_compile funclip/launch.py funclip/videoclipper.py funclip/utils/subtitle_utils.py funclip/llm/openai_api.py → OK
    • git diff --check → clean
    • Full suite: 6 pre-existing env failures (missing twelvelabs/gradio) identical on clean main; zero regressions.
  • I checked the affected README/docs links, if changed.
  • I verified the affected LLM path, if changed: L3 live test with a real key through the actual openai_call() code path → orcarouter/auto → https://api.orcarouter.ai/v1/chat/completions → 200 ORCA-LIVE-OK; _resolve_model_config("orcarouter/auto") resolves base_url https://api.orcarouter.ai/v1 and preserves the full model ID.

Screenshots or clips

No UI visuals — the change adds dropdown entries and a routing branch. The dropdown now lists orcarouter/auto, orcarouter/fusion, orcarouter/fusion-flash, orcarouter/fusion-mini.

Notes for reviewers

  • OrcaRouter is a smart-routing gateway: model IDs must keep the orcarouter/ prefix (a bare auto is not routable), so unlike AtlasCloud/MiniMax the prefix is intentionally preserved in the wire request.
  • OrcaRouter only exposes chat completions; no embeddings/audio endpoints, so no other FunClip surface was touched.
  • Updated (2026-08-27): clarified in both READMEs that enforcement follows the attached policy, applies only to traffic traversing the gateway, and that the documented tight posture enables default-deny; added links to the security quickstart, Guardrails, and Firewall docs.

Disclosure: I'm an engineer on the OrcaRouter team.

@claude
feat(llm): add OrcaRouter as a named OpenAI-compatible gateway
c30aa39 
Add a named OrcaRouter provider to the transcript-driven LLM clipping
routing, mirroring the existing atlascloud/minimax gateways. Selecting
any orcarouter/ model in the LLM Model Name dropdown routes requests to
https://api.orcarouter.ai/v1/chat/completions with the ORCAROUTER_API_KEY
environment variable (or pasted APIKEY).

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: XiaoHuo888-hue <jinhao.song@myflashcloud.com>
LauraGPT
LauraGPT requested changes Aug 25, 2026
View reviewed changes

@LauraGPT LauraGPT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Focused integration tests passed (18/18), and the four documented model slugs are valid. One blocking documentation issue remains: the two READMEs describe optional security controls as active by default.

Comment thread README.md Outdated

Besides the transcript-based LLMs above, FunClip can route LLM-assisted clipping through [OrcaRouter](https://www.orcarouter.ai), an OpenAI-compatible smart-routing gateway. Select any `orcarouter/` model in the **LLM Model Name** dropdown (`orcarouter/auto` routes each request to the best model for the task), paste an OrcaRouter API key in the **APIKEY** box, and click 'LLM Inference' — FunClip sends the transcript and prompts to `https://api.orcarouter.ai/v1/chat/completions`, and the returned segments work with the existing 'AI Clip' button unchanged.

OrcaRouter exposes one endpoint for all frontier and open-weight models, so you can switch routing targets without changing FunClip. It also runs gateway-level, zero-trust security for AI agents on the same endpoint — screening every prompt/response and governing every tool call on a default-deny basis, with no application code changes.

@LauraGPT LauraGPT Aug 25, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Make the security guarantee conditional on configuration

This integration only supplies the base URL, API key, and model; it does not attach a Guardrail or Firewall policy or apply the tight autonomy posture. OrcaRouter documents that with neither a key attachment nor a workspace default there is no enforcement, and that default-deny requires explicitly applying tight. It also scopes Firewall coverage to tool calls that cross or report through the gateway, not every tool call. As written, users can reasonably believe selecting OrcaRouter protects every prompt, response, and action automatically. Please qualify this paragraph in both READMEs, link the security setup, and avoid the unconditional every-tool-call claim.

References: https://docs.orcarouter.ai/features/guardrails#scoping-and-the-workspace-default, https://docs.orcarouter.ai/features/firewall#scoping-and-resolution, https://docs.orcarouter.ai/security/concepts/quickstart, https://docs.orcarouter.ai/security/concepts/shared-responsibility

docs: qualify OrcaRouter security enforcement as opt-in
caa6b3a 
Address reviewer feedback on PR modelscope#205: the integration only supplies the
base URL, API key, and model, and does not attach a Guardrail or Firewall
policy by itself. Make clear enforcement is opt-in once a policy is
applied, and scope coverage to tool calls that traverse the gateway.
Add links to the security quickstart, Guardrails, and Firewall docs in
both README.md and README_zh.md.

Signed-off-by: XiaoHuo888-hue <jinhao.song@myflashcloud.com>
@XiaoHuo888-hue

XiaoHuo888-hue commented Aug 26, 2026

Copy link
Copy Markdown
Author

Thanks for the review! Addressed in commit caa6b3a:

  • Both README.md and README_zh.md now state that enforcement is opt-in: this integration only supplies the base URL, API key, and model, and does not attach a Guardrail or Firewall policy by itself. It no longer claims that selecting OrcaRouter protects every prompt/response/action automatically, and no longer uses an unconditional "every tool call" phrasing — coverage is scoped to tool calls that traverse the gateway.
  • Added links to the security quickstart, Guardrails scoping, and Firewall scoping docs, and mirrored the same qualified wording in the PR summary.
  • No code paths changed (docs-only fix); the previously verified test results and L3 live test remain valid.

Thanks again for the careful read.

LauraGPT
LauraGPT requested changes Aug 26, 2026
View reviewed changes

@LauraGPT LauraGPT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The opt-in and gateway-scope clarifications now address most of the review.

One accuracy issue remains in both READMEs: “once a policy is attached ... on a default-deny basis” implies that attaching any policy enables default-deny. The linked quickstart distinguishes the baseline posture (audit/flag with limited denial) from tight, which is the documented default-deny posture.

Please change this to say enforcement follows the attached policy, and mention that tight enables default-deny. Please also apply the same scoping to the PR description, which still says “every prompt/response” and “every tool call”. After that wording-only correction, I recommend approval.

@LauraGPT
docs: clarify OrcaRouter policy enforcement
613ca5e 
Signed-off-by: zhifu gao <18321252+LauraGPT@users.noreply.github.com>
LauraGPT
LauraGPT previously approved these changes Aug 27, 2026
View reviewed changes

@LauraGPT LauraGPT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The remaining documentation blocker is resolved on signed+DCO head : enforcement now follows the attached scoped policy, gateway coverage is bounded to traversing traffic, and the documented posture is identified as the default-deny option in both READMEs and the PR description. Fresh focused verification passed (8 OrcaRouter tests; 11 OrcaRouter plus generic OpenAI routing tests), along with Python compile and diff checks. Approved.

@LauraGPT
LauraGPT dismissed their stale review August 27, 2026 13:02

Superseded to restore two literals removed by shell quoting.

LauraGPT
LauraGPT approved these changes Aug 27, 2026
View reviewed changes

@LauraGPT LauraGPT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The remaining documentation blocker is resolved on signed+DCO head 613ca5e7ba6330d53602552e389b5a4b5eb175a7: enforcement now follows the attached scoped policy, gateway coverage is bounded to traversing traffic, and the documented tight posture is identified as the default-deny option in both READMEs and the PR description. Fresh focused verification passed (8 OrcaRouter tests; 11 OrcaRouter plus generic OpenAI routing tests), along with Python compile and diff checks. Approved.

@LauraGPT
LauraGPT merged commit ff032af into modelscope:main Aug 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@LauraGPT LauraGPT LauraGPT approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@XiaoHuo888-hue @LauraGPT